Imaging device provided with light source that emits pulsed light and image sensor

ABSTRACT

An optical filter including filter regions arrayed two-dimensionally, in which the filter regions include a first region and a second region; a wavelength distribution of an optical transmittance of the first region has a first local maximum in a first wavelength band and a second local maximum in a second wavelength band that differs from the first wavelength band, and a wavelength distribution of an optical transmittance of the second region has a third local maximum in a third wavelength band that differs from each of the first wavelength band and the second wavelength band and a fourth local maximum in a fourth wavelength band that differs from the third wavelength band.

CROSS REFERENCE

This application is the Continuation of U.S. Patent Application Ser. No.16/245,107 filed Jan. 10, 2019, which is the Continuation of U.S.Application Ser. No. 15/188,071, now U.S. Pat. No. 10,215,636, filed onJun. 21, 2016, which claims the benefit of Japanese Application No.2015-133891 filed on Jul. 2, 2015, the entire contents of each arehereby incorporated by reference.

BACKGROUND 1. Technical Field

The present disclosure relates to an imaging device that acquiresinternal information of a target such as a gas or a living body.

2. Description of the Related Art

In applications such as the detection of gas leakages, biometry, andmaterial analysis, methods are used in which light (including visiblelight, infrared rays, or ultraviolet rays) is irradiated onto a target,and the transmitted light, reflected light, or scattered light therefromis detected to thereby acquire internal information of the target.

Examples of imaging systems in which such methods are used are disclosedin Japanese Unexamined Patent Application Publication No. 2005-91343,Japanese Unexamined Patent Application Publication No. 2008-149154,International Publication No. 2013/002350, and the specification of U.S.Pat. No. 7,283,231, for example.

SUMMARY

In one general aspect, the techniques disclosed here feature an opticalfilter including filter regions arrayed two-dimensionally, wherein thefilter regions include a first region and a second region; a wavelengthdistribution of an optical transmittance of the first region has a firstlocal maximum in a first wavelength band and a second local maximum in asecond wavelength band that differs from the first wavelength band, anda wavelength distribution of an optical transmittance of the secondregion has a third local maximum in a third wavelength band that differsfrom each of the first wavelength band and the second wavelength bandand a fourth local maximum in a fourth wavelength band that differs fromthe third wavelength band.

Additional benefits and advantages of the disclosed embodiments willbecome apparent from the specification and drawings. The benefits and/oradvantages may be individually obtained by the various embodiments andfeatures of the specification and drawings, which need not all beprovided in order to obtain one or more of such benefits and/oradvantages.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a drawing schematically depicting a configuration of animaging device in embodiment 1 of the present disclosure;

FIG. 2A is a drawing depicting an example of a two-dimensionaldistribution of light transmittance of an encoding element in embodiment1 of the present disclosure;

FIG. 2B is a drawing depicting another example of a two-dimensionaldistribution of light transmittance of the encoding element inembodiment 1 of the present disclosure;

FIG. 3 is a drawing depicting a schematic configuration example of onepixel of an image sensor in embodiment 1 of the present disclosure;

FIG. 4 is a flowchart depicting an imaging operation in embodiment 1 ofthe present disclosure;

FIG. 5 is a drawing describing a relationship between a timing at whichpulsed light is emitted by a light source, timings at which Ramanscattered light due to detection-target gas molecules reaches the imagesensor, and timings at which light is received in floating diffusionlayers of the image sensor in embodiment 1 of the present disclosure;

FIG. 6 is a drawing depicting an example of an overall configuration ofthe image sensor in embodiment 1 of the present disclosure;

FIG. 7 is a conceptual diagram depicting an impression of spectralseparation processing in embodiment 1 of the present disclosure;

FIG. 8 is a drawing depicting an overview of 3D image generationprocessing by a second signal processing unit in embodiment 1 of thepresent disclosure;

FIG. 9 is a drawing depicting a configuration of embodiment 2 of thepresent disclosure;

FIG. 10 is a drawing schematically depicting a configuration of anencoding spectroscopic element in embodiment 2 of the presentdisclosure;

FIG. 11A is a drawing for describing spectral transmittancecharacteristics in a certain region of the encoding spectroscopicelement in embodiment 2 of the present disclosure;

FIG. 11 B is a drawing depicting results of averaging the spectraltransmittance depicted in FIG. 11A in each wavelength band;

FIG. 12 is a drawing depicting a configuration of an imaging device inembodiment 3 of the present disclosure;

FIG. 13 is a drawing depicting an overview of a pixel configuration ofan image sensor in embodiment 3 of the present disclosure;

FIG. 14 is a drawing depicting timings of a light-emission pulse,diffused light of each distance range, signal accumulation pulses toeight floating diffusion layers, and a discharge pulse to a drain inembodiment 3 of the present disclosure;

FIG. 15 is a drawing depicting an example of an overall configuration ofthe image sensor in embodiment 3 of the present disclosure;

FIG. 16A is a graph depicting the wavelength dependency of absorptioncoefficients for oxidized hemoglobin, deoxidized hemoglobin, and water;

FIG. 16B is a graph depicting the wavelength dependency of thescattering coefficient of light within biological tissue;

FIG. 17 is a flowchart depicting a signal processing flow in embodiment3 of the present disclosure;

FIG. 18 is a drawing depicting an example of changes over time in theamount of light that reaches one pixel in embodiment 3 of the presentdisclosure; and

FIG. 19 is a drawing depicting a configuration of an imaging device inembodiment 4 of the present disclosure.

DETAILED DESCRIPTION (Findings Forming the Basis for the PresentDisclosure)

Prior to describing embodiments of the present disclosure, the findingsforming the basis for the present disclosure will be described.

The present inventors discovered the following problems with regard toconventional gas leakage detection systems and biological informationdetection systems.

In a conventional common gas leakage detection system, suctioned gas isbrought into direct contact with a sensor, and the concentration of adetection-target gas is measured on the basis of changes in physicalquantities such as electric resistance and current values within thesensor. However, in a method such as this, it is necessary to bring agas into direct contact with the sensor, and it is therefore notpossible to detect a gas in a location away the sensor. In addition,there is a drawback in that there are many uncertainties due toenvironmental conditions such as the wind direction and the installationlocation.

Japanese Unexamined Patent Application Publication No. 2005-91343discloses an example of a gas detection system with which this kind ofdrawback is resolved. In the system of Japanese Unexamined PatentApplication Publication No. 2005-91343, ultraviolet laser light isirradiated onto a gas to cause Raman scattering. Raman scattering is aphenomenon in which scattered light having a wavelength that isdifferent from the wavelength of the light incident on molecules isgenerated. This phenomenon occurs due to the molecules receiving lightand entering a temporary high-energy intermediary state, and thereaftertransitioning to a vibrationally excited state or a ground state. In thecase where a transition is made from the ground state to thevibrationally excited state via the intermediary state, scattered lighthaving a longer wavelength than the incident light is generated. Thiswavelength shift is called a Stokes shift. Conversely, in the case wherea transition is made from the vibrationally excited state to the groundstate via the intermediary state, scattered light having a shorterwavelength than the incident light is generated. This wavelength shiftis called an anti-Stokes shift.

In the system of Japanese Unexamined Patent Application Publication No.2005-91343, a leakage of a gas and the type thereof are detected in anon-contact manner by detecting the Stokes shift of Raman scatteredlight. A gas leakage can be displayed as a two-dimensional image bysuperimposing an image depicting the distribution of the detected gasonto a visible image of a monitoring-target space. It is described thatit is thereby possible to safely monitor gas leakages and specifyleakage locations remotely.

Japanese Unexamined Patent Application Publication No. 2005-91343describes that a photographing timing is changed according to thedistance range of a detection target. Photographing at a single distancerange is possible by means of brief exposure synchronized with lightemission by a light source. However, it is not possible to performphotographing at a plurality of distance ranges at the same time.

In the embodiments of the present disclosure, it is possible to performphotographing at a plurality of distance ranges using a method referredto as “time-resolved imaging”. Hereinafter, the time-resolved imagingused in the embodiments of the present disclosure will be described.

With the progress of laser light irradiation technology, it has becomepossible to repeatedly irradiate a target with extremely brief pulsedlight of the order of nanoseconds (ns) to picoseconds (ps), and tocontrol, at high speed, the light reception time of an image sensor insynchronization therewith. This kind of imaging that is performed with alight source and an image sensor being controlled at high speed isreferred to as “time-resolved imaging”. As a representative applicationexample, measurement imaging devices that use a method referred to astime of flight (TOF) in which the distance to a subject is detected inpixel units on the basis of the flight time of light have beencommercialized.

Due to the application of time-resolved imaging, for phenomena thatrepeatedly occur in a similar manner when light is emitted, it has alsobecome possible for extremely brief light reception to be repeated inconjunction with light emission to accumulate signals, and for briefchanges in phenomena to be reproduced as video. For example, it has alsobecome possible to visualize, as video, the way in which lightpropagates.

Time-resolved imaging technology is being considered for application notonly for spaces through which light is transmitted but also for lightscattering bodies such as living bodies. For example, in time-resolveddiffuse optical tomography, a near infrared ray of two wavelengthshaving a relatively low light absorption rate with respect to both waterand hemoglobin is used to form a tomographic image from theconcentration distribution of oxidized hemoglobin and deoxidizedhemoglobin within a living body.

Particularly in the case where a living body is to be a target,noninvasive biological observation without any exposure to radiationwhatsoever is possible by using infrared rays (also referred to as“infrared light” hereinafter) that are safe for living bodies, comparedwith roentgen and X-ray CT widely used as diagnostic imaging methods.Furthermore, imaging by near-infrared light is suitable for reducingdevice size and weight and reducing cost compared with an MRI device,for example, for which nuclear magnetic resonance is used. Thus, imagingby near-infrared light has begun to be widely used in cutting-edgeresearch relating to elucidating the functions of biological tissue.

However, near-infrared light scatters to a considerably high degree whenpassing through biological tissue, and as a result, severe lightscattering occurs within the biological tissue. There is a problem inthat this light scattering causes a considerable decline in the spatialresolution of imaging.

One known method for solving this problem is an image reconstructionalgorithm in which: an optical characteristic value (absorptioncoefficient and scattering coefficient, for example) spatialdistribution serving as a scattering body is assumed fordetection-target biological tissue to numerically calculate lightpropagation within the scattering body (forward problem); and the resultthereof and data actually obtained by time-resolved imaging arecompared, and the assumption of the spatial distribution is repeateduntil both match (inverse problem). By using an algorithm such as this,it is possible to reconstruct a three-dimensional image having improvedspatial resolution.

Conventional time-resolved diffuse optical tomography is useful inapplications such as research regarding activation of the brain andoptical mammography for screening for breast cancer. However, a lightsource of two or three restricted wavelengths is used, and it istherefore not possible to acquire spectral information of a large numberof bands. Consequently, it is not possible to detect information ofmolecules having high specificity from living tissue.

Japanese Unexamined Patent Application Publication No. 2008-149154discloses an example of a system that detects information of moleculeshaving high specificity from living tissue. The system disclosed inJapanese Unexamined Patent Application Publication No. 2008-149154includes: a near-infrared light source that provides incident light; amultipoint incident light irradiation array for guiding light into aliving body from two or more separate excitation points; a plurality ofoptical fibers for transmitting light from the light source to eachpoint of the multipoint incident light irradiation array; a multipointdetection array for collecting fluorescence emitted from a target fromtwo or more separate collection points; a two-dimensional light emissionarray for transmitting light emitted from the target to a detector; aplurality of optical fibers for transmitting light from each collectionpoint to a corresponding point on the two-dimensional light emissionarray; and the detector for detecting the light emitted from each pointof the two-dimensional light emission array and converting into adigital signal corresponding to the light emitted from the target. It isdescribed that, according to this kind of configuration, it is possibleto acquire, with a high degree of high sensitivity, three-dimensionallocation information of abnormal molecules constituting the cause of adisease.

However, Japanese Unexamined Patent Application Publication No.2008-149154 does not describe an imaging method with which informationof a plurality of distance ranges is acquired with single-frame imaging.

The imaging device according to the embodiments of the presentdisclosure acquires information of a plurality of distance ranges withsingle-frame imaging, and also acquires information of a plurality ofwavelengths. A technique referred to as compressed sensing is used toacquire information of a plurality of wavelengths. Hereinafter,compressed sensing will be described.

By utilizing spectral information of a large number of bands (severaltens of bands or more, for example) that are each a narrow band, it ispossible to comprehend detailed physical properties of an observationobject not possible with a conventional RGB image. A camera thatacquires this multi-wavelength information is referred to as a“hyperspectral camera”. Hyperspectral cameras are used in various fieldssuch as food inspection, biological inspection, pharmaceutical productdevelopment, and mineral component analysis. For example, InternationalPublication No. 2013/002350 discloses a device that distinguishesbetween tumorous sites and non-tumorous sites of a subject by generatingan image acquired with the wavelengths of the observation target beingrestricted to narrow bands. This device, by irradiating excitationlight, detects the emission of 635-nm fluorescence by protoporphyrin IXthat is accumulated in cancer cells, and the emission of 675-nmfluorescence by photo-protoporphyrin. Tumorous sites and non-tumoroussites are thereby identified.

The specification of U.S. Pat. No. 7,283,231 discloses an example of ahyperspectral camera in which compressed sensing is used. A devicedisclosed in the specification of U.S. Pat. No. 7,283,231 diffractslight from a measurement target with a first spectroscopic element suchas a prism, and then implements marking with an encoding mask, and, inaddition, restores the path of light rays with a second spectroscopicelement. Thus, an image that has been encoded and multiplexed withrespect to the wavelength axis is acquired by a sensor. By applyingcompressed sensing, a plurality of multi-wavelength images arereconstructed from the multiplexed image.

Compressed sensing is a technique for restoring a larger amount of datafrom acquired data of which there is a small number of samples. If thetwo-dimensional coordinates of a measurement target are taken as (x, y)and the wavelength is taken as λ, the desired data f isthree-dimensional data of x, y, λ. In contrast to this, image data gobtained by a sensor is two-dimensional data that has been compressedand multiplexed in the λ axis direction. The problem of obtaining thedata f having a relatively large data amount from the acquired image ghaving a relatively small data amount is what is referred to as anill-posed problem, and cannot be solved as it is. However, natural imagedata generally has redundancy, and by using this skillfully it ispossible to convert this ill-posed problem into a well-posed problem.JPEG compression is an example of a technique for reducing the amount ofdata by using the redundancy of an image. JPEG compression uses a methodin which image information is converted into frequency components, andnon-essential portions of the data such as portions having low visualrecognizability are removed. In compressed sensing, this kind oftechnique is incorporated into calculation processing, and desired dataspaces are converted into spaces indicated by redundancy to therebyreduce unknowns and obtain a solution. For this conversion, a discretecosine transform (DCT), wavelet transform, Fourier transform, totalvariation (TV), or the like is used.

The specification of U.S. Pat. No. 7,283,231 discloses a compressedsensing technique such as the aforementioned, but does not disclose amethod for simultaneously acquiring information of a plurality ofwavelengths, spatial distribution information, and information of aplurality of times (distance ranges).

With conventional techniques, it has been extremely difficult tosimultaneously perform time-resolved imaging and spectroscopic imagingwithout sacrificing spatial resolution. An imaging device with whichthese are simultaneously achieved does not exist at the present time. Inorder to acquire information of a plurality of wavelengths, there hasonly been a method in which light sources having different lightemission wavelengths are made to irradiate in a time-divided manner, anda method in which detectors are separated for each wavelength of adetection target. Simultaneity is sacrificed with the former, andspatial resolution is sacrificed with the latter.

The present inventors discovered the aforementioned problems in theconventional techniques, and carried out a diligent investigation intomethods for solving these problems. As a result, it was found that itbecomes possible to acquire information of a plurality of distanceranges and information of a plurality of wavelengths without sacrificingspatial resolution by using: a light source that emits pulsed lightincluding components of a plurality of wavelengths; a time-resolvedimage sensor that detects light that is incident from a target, at highspeed and in a time-divided manner; and an encoding element thatmodulates the intensity of the light that is incident on the imagesensor, in accordance with location. If safe infrared light is used withrespect to a living body in particular, imaging that is noninvasive andwith which there is no exposure to radiation such as with X-rays ispossible. Hereinafter, an example of an imaging device having such aconfiguration will be described.

The present disclosure includes imaging devices according to thefollowing items.

-   [Item 1]

An imaging device, provided with:

a light source that, in operation, emits pulsed light includingcomponents of different wavelengths;

an encoding element that has regions each having different lighttransmittance, through which incident light from a target onto which thepulsed light has been irradiated is transmitted;

a spectroscopic element that, in operation, causes the incident lighttransmitted through the regions to be dispersed into light rays inaccordance with the wavelengths; and

an image sensor that, in operation, receives the light rays dispersed bythe spectroscopic element.

-   [Item 2]

An imaging device, provided with:

a light source that, in operation, emits pulsed light includingcomponents of different wavelengths;

an encoding spectroscopic element that has regions each having differentwavelength distributions of light transmittance, through which incidentlight from a target onto which the pulsed light has been irradiated istransmitted; and

an image sensor that, in operation, receives the incident lighttransmitted through the regions.

-   [Item 3]

The imaging device according to item 1 or 2, further comprising:

a control circuit, wherein:

the target has a first portion and a second portion; and

the control circuit,

at a first time, causes the light source to emit the pulsed light,

at a second time subsequent to the first time, causes the image sensorto accumulate first signal charge that is based upon first incidentlight from the first portion of the target,

at a third time subsequent to the second time, causes the image sensorto accumulate second signal charge that is based upon second incidentlight from the second portion of the target,

and causes the image sensor to output a first image signal that is basedupon the first signal charge and a second image signal that is basedupon the second signal charge.

-   [Item4]

The imaging device according to item 3, wherein:

the image sensor includes photodetection cells that each include a firstcharge accumulator and a second charge accumulator; and

the control circuit,

at the second time, causes the first charge accumulator in each of thephotodetection cells to accumulate the first signal charge,

and, at the third time, causes the second charge accumulator in each ofthe photodetection cells to accumulate the second signal charge.

-   [Item5]

The imaging device according to item 3 or 4,

wherein the control circuit,

after causing the light source and the image sensor to repeat, more thanonce, the emitting of the pulsed light at the first time, theaccumulating of the first signal charge at the second time, and theaccumulating of the second signal charge at the third time,

causes the image sensor to output the first image signal and the secondimage signal.

-   [Item6]

The imaging device according to any of items 3 to 5, further providedwith:

a signal processing circuit that, in operation, separates the firstimage signal into first separate image signals in accordance with thewavelengths, and separates the second image signal into second separateimage signals in accordance with the wavelengths.

-   [Item7]

The imaging device according to item 6,

wherein, in operation, the signal processing circuit generates athree-dimensional image of the target on the basis of the first separateimage signals and the second separate image signals.

-   [Item8]

The imaging device according to item 7, wherein:

the target is a light scattering body; and,

in operation, the signal processing circuit

assumes an optical characteristic value distribution for the target,

calculates light propagation within the target,

compares a calculation result for the light propagation with the firstseparate image signals and the second separate image signals,

repeatedly assumes the optical characteristic value distribution until acomparison result indicates matching, and

generates the three-dimensional image on the basis of the opticalcharacteristic value distribution from when the comparison resultindicates matching.

-   [Item9]

The imaging device according to any of items 1 to 8,

wherein the target is a gas.

-   [Item10]

The imaging device according to any of items 1 to 7,

wherein the target is a light scattering body.

-   [Item11]

The imaging device according to any of items 1 to 10,

wherein the pulsed light is an ultraviolet ray or an infrared ray.

-   [Item12]

An imaging device, comprising:

a first light source that, in operation, emits first pulsed lightincluding a first wavelength;

a second light source that, in operation, emits second pulsed lightincluding a second wavelength that is different from the firstwavelength;

an encoding element that has regions each having different lighttransmittance, through which incident light from a target onto which thefirst pulsed light and the second pulsed light are irradiated istransmitted;

a spectroscopic element that, in operation, causes the incident lighttransmitted through the regions to be dispersed into first incidentlight including the first wavelength and second incident light includingthe second wavelength; and

an image sensor that, in operation, receives the first incident lightand the second incident light. The imaging device according to item 12may further comprises a control circuit that at a first time, causes thefirst light source to emit the first pulsed light, and, at a second timethat is different from the first time, causes the second light source toemit the second pulsed light.

-   [Item13]

An imaging device, comprising:

a first light source that, in operation, emits first pulsed lightincluding a first wavelength;

a second light source that, in operation, emits second pulsed lightincluding a second wavelength that is different from the firstwavelength;

an encoding spectroscopic element that has regions each having differentwavelength distributions of light transmittance, through which incidentlight from a target onto which the first pulsed light and the secondpulsed light are irradiated is transmitted; and

an image sensor that, in operation, receives the incident lighttransmitted through the regions. The imaging device according to item 13may further comprises a control circuit that, at a first time, causesthe first light source to emit the first pulsed light, and, at a secondtime that is different from the first time, causes the second lightsource to emit the second pulsed light.

In the present disclosure, all or part of any of a unit, device, memberor portion, or all or part of the functional blocks in the blockdiagrams may be implemented as one or more electronic circuits includinga semiconductor device, a semiconductor integrated circuit (IC) or alarge scale integration (LSI). The LSI or IC may be integrated into onechip, or may be a combination of a plurality of chips. For example,functional blocks other than a storage element may be integrated intoone chip. The name used here is LSI or IC, but it may also be referredto as a system LSI, a very large scale integration (VLSI), or an ultralarge scale integration (ULSI) depending on the degree of integration. Afield-programmable gate array (FPGA) that can be programmed aftermanufacturing an LSI, or a reconfigurable logic device that allowsreconfiguration of the connection relationship inside the LSI or thesetup of circuit cells inside the LSI can also be used for the samepurpose.

In addition, it is also possible for the functions or operations of allor part of the unit, device, member or portion to be implemented bymeans of software processing. In such a case, the software is recordedon one or more non-transitory recording mediums such as a ROM, anoptical disk or a hard disk drive, and when the software is executed bya processor, the software causes the processor together with peripheraldevices to execute the functions specified in the software. A system ordevice may be provided with such one or more non-transitory recordingmediums on which the software is recorded, a processor, and necessaryhardware devices such as an interface.

Hereafter, embodiments of the present disclosure will be described indetail with reference to the drawings. It should be noted that theembodiments described hereinafter all represent comprehensive orspecific examples. The numerical values, the shapes, the materials, theconstituent elements, the arrangement of the constituent elements, themode of connection, the steps, and the order of the steps and so forthgiven in the following embodiments are examples and are not intended torestrict the present disclosure. The various aspects described in thepresent specification may be combined with each other provided there areno resulting inconsistencies. Furthermore, constituent elements that arenot described in the independent claims indicating the most significantconcepts from among the constituent elements in the followingembodiments are described as optional constituent elements. In thefollowing description, constituent elements having substantially thesame functions are denoted by common reference numerals, anddescriptions thereof have been omitted.

Embodiment 1

FIG. 1 is a drawing schematically depicting a configuration of animaging device in embodiment 1 of the present disclosure. FIG. 1 alsodepicts a detection target 101 as well as other constituent elements ofthe imaging device. This imaging device is able to detect the type andconcentration of a gas constituting the detection target 101, andgenerate a three-dimensional (3D) image of the spatial distribution ofthe concentration of the gas.

The imaging device is provided with: a light source 107 that irradiatesan ultraviolet ray having two wavelength components toward the target101; an optical system (an image-forming optical system 102, anexcitation light cut filter 111, an encoding element 104, a relayoptical system 103, and a spectroscopic element 105) that is arranged inthe optical path of light that is incident from the target 101; an imagesensor 106 that detects light that has passed through the spectroscopicelement 105; and a control circuit 108 that controls the light source107 and the image sensor 106. A signal processing circuit 112 thatprocesses image signals that are output from the image sensor 106 isalso drawn in FIG. 1. The signal processing circuit 112 may beincorporated into the imaging device, or may be a constituent element ofa signal processing device that is electrically connected in a wired orwireless manner to the imaging device. In FIG. 1, each of theimage-forming optical system 102 and the relay optical system 103 isdrawn as a single lens, but they may be an assembly of a plurality oflenses.

The light source 107 in the present embodiment is a laser light sourcethat emits pulsed light of the ultraviolet region having two wavelengthcomponents. The light source 107 repeatedly emits short-pulseultraviolet light rays in accordance with control signals that are inputfrom the control circuit 108. These short-pulse ultraviolet light raysexcite the target 101 (gas) within a detection target space, and causethe generation of Raman scattered light that accompanies wavelengthshifts corresponding to gas molecules. At such time, distinctive RamanStokes light is generated in the gas molecules that have shifted to thelong wavelength side compared with the wavelength of the incidentultraviolet light rays (also called “excitation light”).

The wavelength shift of the Raman Stokes light of the gas molecules isdisclosed in Japanese Unexamined Patent Application Publication No.2005-91343, and is known to have the numerical values given in Tablebelow. In the present embodiment, a description is given of an exampleof the case where the wavelengths of laser pulsed light emitted from thelight source 107 have been set to 355 nm and 266 nm. It should be notedthat these wavelengths are examples, and other wavelengths may be used.Furthermore, the light source 107 may be configured so as to emit pulsedlight having three or more wavelength components.

TABLE RAMAN RAMAN SCATTERING SCATTERING WAVELENGTH WAVELENGTH RAMAN (nm)WHEN LASER (nm) WHEN LASER SHIFT WAVELENGTH WAVELENGTH IS MOLECULE(cm⁻¹) IS 355 nm 266 nm CO₂ 1286 372.0 275.4 CO₂ 1388 373.4 276.2 O₂1556 375.8 277.5 CO 2145 384.3 282.1 N₂ 2331 387.0 283.6 H₂S 2611 391.3285.9 CH₄ 2914 396.0 288.4 CH₄ 3020 397.6 289.2 NH₃ 3334 402.7 291.9 H₂O3652 407.9 294.6 H₂ 4160 416.5 299.1

The light emitted from the light source 107 in the present embodiment isin the ultraviolet wavelength band (approximately 10 nm to approximately400 nm); however, it should be noted that the wavelength of the lightfrom the light source 107 may be appropriately selected according touse. For example, as described in embodiment 3, in an imaging devicethat measures biological tissue, the near-infrared wavelength band(approximately 700 nm to approximately 2500 nm) may be used. Inaddition, it is also possible to use the visible light wavelength band(approximately 400 nm to approximately 700 nm), mid-infrared rays,far-infrared rays, or electromagnetic waves of the radio wave band suchas terahertz waves or millimeter waves. In the present specification,not only visible light but also invisible light such as near-ultravioletrays, near-infrared rays, and radio waves are referred to as “light” forconvenience.

The imaging device of the present embodiment detects Raman Stokes lightgenerated from gas molecules due to the irradiation of pulsed light, bymeans of the image sensor 106 via the optical system. At such time,detection is performed a plurality of times at high speed at differenttimings by means of highly time-resolved imaging. Thus, an image havinga plurality of wavelength components superimposed thereon is acquiredfor each distance range of the target 101. Signals indicating an imagefor each of the distance ranges (sometimes referred to as an “imagesignal”) are sent to the signal processing circuit 112 and processedthereby. The signal processing circuit 112 has a first signal processingunit 109 and a second signal processing unit 110. The first signalprocessing unit 109 generates a plurality of new image signals that areobtained by separating the image signals for each distance range intoeach wavelength component. In the present specification, this processingis sometimes referred to as “spectral separation processing”. The secondsignal processing unit 110 generates three-dimensional image data fromthe image signals that have been separated for each wavelengthcomponent. In a certain aspect, the first signal processing unit 109 andthe second signal processing unit 110 may be realized as separatemodules within the signal processing circuit 112. The first signalprocessing unit 109 and the second signal processing unit 110 may berealized by one processor executing different image processing programs.Details of the processing performed by the signal processing circuit 112are described later on.

Hereinafter, details of the constituent elements will be described.

The control circuit 108 may be an integrated circuit such as a centralprocessing unit (CPU) or a microcomputer. The control circuit 108executes a control program recorded in a memory that is not depicted,for example, to thereby perform control in the form of a lightinginstruction for the light source 107, an imaging instruction for theimage sensor 106, and a calculation instruction for the signalprocessing circuit 112, for example.

The signal processing circuit 112 is a circuit that processes the imagesignals that are output from the image sensor 106. The signal processingcircuit 112 may be realized by a digital signal processor (DSP), aprogrammable logic device (PLD) such as a field-programmable gate array(FPGA), a combination of a central processing unit (CPU) and a computerprogram, or a combination of a graphics processing unit (GPU) and acomputer program, for example. It should be noted that the controlcircuit 108 and the signal processing circuit 112 may be realized bymeans of one integrated circuit.

In the imaging device, the Raman Stokes light generated from the gasmolecules is collected by the image-forming optical system 102, andformed into an image on an image formation surface. At such time,wavelength components produced by Rayleigh scattering or the like ofpulsed light having two wavelengths that become noise are cut by theexcitation light cut filter 111.

The encoding element 104 is arranged on the image formation surface ofthe image-forming optical system 102. The encoding element 104 is a maskhaving a spatial distribution of light transmittance, and has aplurality of regions having different light transmittance arrayedtwo-dimensionally. More specifically, there are a plurality of regionshaving a first light transmittance, and a plurality of regions having asecond light transmittance that is lower than the first lighttransmittance. The encoding element 104 transmits the incident lightwith the intensity thereof being modulated in accordance with location.This process performed by the encoding element 104 is referred to as“encoding”.

FIG. 2A is a drawing depicting an example of a two-dimensionaldistribution of light transmittance of the encoding element 104. In FIG.2A, the black portions represent regions through which light is mostlynot transmitted (referred to as “light-blocking regions”), and the whiteportions represent regions through which light is transmitted (referredto as “light-transmitting regions”). In this example, the lighttransmittance of the light-transmitting regions is approximately 100%,and the light transmittance of the light-blocking regions isapproximately 0%. The encoding element 104 is divided into a pluralityof rectangular regions, and each rectangular region is alight-transmitting region or a light-blocking region. Thetwo-dimensional distribution of the light-transmitting regions and thelight-blocking regions in the encoding element 104 may be a randomdistribution or a quasi-random distribution, for example.

The thinking behind a random distribution and a quasi-randomdistribution is as follows. First, the rectangular regions in theencoding element 104 are deemed to be vector elements having values of 1or 0, for example, in accordance with the light transmittance. In otherwords, a set of the rectangular regions arranged side-by-side in a rowis deemed to be a multidimensional vector having values of 1 or 0.Consequently, the encoding element 104 is provided with a plurality ofmultidimensional vectors in the row direction. At such time, a randomdistribution means that any two multidimensional vectors are independent(not parallel). Furthermore, a quasi-random distribution means thatnon-independent configurations are included among some of themultidimensional vectors.

The encoding process performed by the encoding element 104 can be saidto be a process for performing marking for distinguishing between imagesproduced by light of each wavelength diffracted by the subsequentspectroscopic element 105. Provided that such marking is possible, thetransmittance distribution may be set in an arbitrary manner. In theexample depicted in FIG. 2A, the ratio between the number of blackportions and the number of white portions is 1:1, but there is norestriction to such a ratio. For example, the distribution may be biasedin one way, such as the number of white portions to the number of blackportions being 1:9. The encoding element 104 schematically depicted inFIG. 1 has more (wider) light-transmitting regions than light-blockingregions.

FIG. 2B is a drawing depicting another example of a two-dimensionaldistribution of light transmittance of the encoding element 104. Asdepicted in FIG. 2B, the encoding element 104 may be a mask having agrayscale transmittance distribution. A grayscale transmittancedistribution means a distribution that includes regions having anintermediate transmittance in which the transmittance is greater than 0%and less than 100%. Such an encoding element 104 has a plurality ofrectangular regions having light transmittances that are different fromthe first and second light transmittances. Information regarding thetransmittance distribution of the encoding element 104 is acquired inadvance by means of design data or actual measured calibrations, and isused in signal processing that is described later on.

Reference will once again be made to FIG. 1. The spectroscopic element105 in the present embodiment is an element that causes an incidentlight beam to be dispersed according to wavelength. The spectroscopicelement 105 is configured from a combination of prisms made up of twomaterials. The two materials are selected from materials in which therefractive indexes for light of a specific wavelength are approximatelythe same and the Abbe numbers for light of that wavelength aredifferent, for example, materials in which the Abbe numbers for light ofthat wavelength deviate. The specific wavelength may be set to arepresentative wavelength (dominant wavelength) within a desiredspectral wavelength range, for example. The dominant wavelength may be acentral wavelength in a measurement-target wavelength range or awavelength that is considered to be important, for example. The Abbenumber “deviating” here means that the difference between the Abbenumbers of the two materials is 10 or more. The difference between theAbbe numbers of the two materials may be 15 or more or may be 20 ormore. The refractive indexes of the two materials being approximatelythe same means that the difference between the refractive indexes of thematerials is 0.05 or less. The refractive indexes of the two materialsbeing approximately the same may mean that the difference between therefractive indexes of the materials is 0.02 or less.

An Abbe number in the present specification is not restricted to an Abbenumber relating to a Fraunhofer line wavelength that is generally used,and may be defined with respect to any wavelength. In the presentdisclosure, an Abbe number vb can be defined as in Math. 1 below withrespect to arbitrary wavelengths λa, λb, and λc that satisfy λa<λb<λc.

$\begin{matrix}{v_{b} = \frac{n_{b} - 1}{n_{a} - n_{c}}} & \left( {{Math}.\mspace{14mu} 1} \right)\end{matrix}$

Here, na, nb, and nc each represent a refractive index in thewavelengths λa, λb, and λc. λa and λc may be any wavelengths. λa and λcmay be a wavelength near the start or end of the used wavelength band.

By making the surface onto which light rays are incident and the surfacefrom which light rays are emitted substantially parallel in thespectroscopic element 105, it is possible to suppress the generation ofcoma aberration. Here, “substantially parallel” is not restricted to thecase where the surfaces are strictly parallel, and includes the casewhere the angle formed by the two surfaces is 3° or less. This angle maybe set to 1° or less or may be set to 0.5° or less. By using thespectroscopic element 105 such as that depicted in FIG. 1, an effect isobtained in that it is possible to reduce deterioration in theresolution of generated images for each wavelength band.

It should be noted that the spectroscopic element 105 does notnecessarily have to be an element having two types of materials such asthose described above joined together. For example, a general prism ordiffraction optical element may be used.

Light encoded by the encoding element 104 is collected by the relayoptical system 103 and input to the spectroscopic element 105. Thespectroscopic element 105 diffracts light such that an optical imageformed on an imaging surface of the image sensor 106 shifts in adirection corresponding to the image vertical direction (the verticaldirection indicated in FIG. 1) in accordance with wavelength. The degreeof that shift (sometimes referred to as the “diffraction amount”) isdetermined according to the refractive indexes, the Abbe numbers, andthe inclination angle of the connecting surfaces of the materials makingup the spectroscopic element 105, and also the distance between thespectroscopic element 105 and the image sensor 106. In the case wherethe spectroscopic element 105 is a diffraction optical element, thediffraction amount can be adjusted by changing the pitch of thediffraction grating. In the present embodiment, the spectroscopicelement 105 diffracts light in the direction corresponding to the imagevertical direction, but may diffract light in the horizontal directionor another direction.

The amount of shift of the image on the imaging surface of the imagesensor 106 due to the spectroscopic element 105 can be calculated inadvance by a calculation based upon the design specifications or bymeans of actual measured calibrations. The spectral shift due to thespectroscopic element 105 is a continuous shift rather than a discreteshift for each measurement-target wavelength band. Meanwhile, in thesignal processing circuit 112, as described later on, a spectralseparation image is reconstructed for each wavelength band of aprescribed width. Therefore, strictly speaking, an image for eachwavelength is shifted on the image sensor 106 even within the wavelengthband for each image that is to be reconstructed. An image shift withinthe wavelength band may be corrected in order to improve the precisionof the reconstruction of a spectral separation image. This correctionmay be performed by means of a computer calculation; however, when theeffects of aberration in the optical system and mounting errors are alsoconsidered, the correction may be performed by means of actual measuredcalibrations. For example, it is possible for calibrations to beperformed by arranging a white board in a prescribed location as asubject, and causing an image of the encoding element 104 to be formedon the image sensor 106 via a band pass filter for a desired wavelengthband. The band pass filter may be switched for each band to acquire dataof all desired bands; however, several bands may be selected andmeasured, and for other bands, calculations may be performed byinterpolation of the measured data. According to this method, it ispossible to calculate the spectral shift amount due to the spectroscopicelement 105, and to also acquire transmittance information of theencoding element 104 for each wavelength band. Elements of a matrix H inMath. 2 described later on are determined on the basis of data of thecalibrations calculated here.

A plurality of separated images having encoding information due to theencoding element 104 are formed on the image formation surface of theimage sensor 106 while being shifted in the vertical direction for eachwavelength, as a multiplex image in which the plurality of imagesoverlap each other. The image sensor 106 captures this multiplex image.At such time, by performing high-speed time-resolved imaging, aplurality of images are acquired for each distance range of the target101. Hereinafter, a configuration and operation of the image sensor 106will be described.

The image sensor 106 has a plurality of photodetection cells (alsoreferred to as “pixels” in the present specification) arrayedtwo-dimensionally on the imaging surface. The photodetection cells havea plurality of charge accumulation units (floating diffusion layers, forexample).

FIG. 3 is a drawing depicting a schematic configuration example of onepixel 201 of the image sensor 106. It should be noted that FIG. 3depicts the configuration of one pixel 201 in a schematic manner, anddoes not necessarily reflect the actual construction. The pixel 201 has:a photoelectric conversion unit (photodiode) 203 that performsphotoelectric conversion; four floating diffusion layers (FD) 204 to 207that accumulate signal charge; and a signal charge discharge unit(drain) 202 that discharges signal charge.

Photons that are incident on each pixel due to one emission of pulsedlight are converted into signal electrons (signal charge) by thephotodiode 203. The converted signal electrons are discharged to thedrain 202 or divided into any of the floating diffusion layers 204 to207 in accordance with control signals that are input from the controlcircuit 108.

The emission of pulsed light from the light source 107, the accumulationof signal charge to the first floating diffusion layer (FD1) 204, thesecond floating diffusion layer (FD2) 205, the third floating diffusionlayer (FD3) 206, and the fourth floating diffusion layer (FD4) 207, andthe discharge of signal charge to the drain 202 are repeatedly performedin this order. This repeated operation is high speed, and, for example,is able to be repeated several ten thousand times to several hundredmillion times within the time of one video frame (approximately 1/30second, for example). Ultimately, the image sensor 106 generates andoutputs four image signals that are based upon the signal chargeaccumulated in the four floating diffusion layers 204 to 207.

FIG. 4 is a flowchart depicting the flow of this operation. The controlcircuit 108, first, at a first time, causes the light source 107 to emitpulsed light that includes a plurality of wavelength components (stepS101). At a subsequent second time, signal charge that is based uponlight that is incident from a first portion of the target 101 isaccumulated in the first floating diffusion layer 204 in the imagesensor 106 (step S102). At a subsequent third time, signal charge thatis based upon light that is incident from a second portion of the target101 is accumulated in the second floating diffusion layer 205 in theimage sensor 106 (step S103). Thereafter, signal charge is sequentiallyaccumulated in the third floating diffusion layer 206 and the fourthfloating diffusion layer 207 in a similar manner (steps S104 and S105).Next, signal charge is discharged to the drain 202 (step S106). Next,the control circuit 108 determines whether or not the number of timesthat the aforementioned signal accumulation cycle has been executed hasreached a prescribed number of times (step S107). In the case where thisdetermination is no, the steps S101 to S107 are repeated until adetermination of yes is made. When a determination of yes is made instep S107, the control circuit 108 causes the image sensor 106 togenerate and output an image signal that is based upon the signal chargeaccumulated in the floating diffusion layers (step S108).

Hereinafter, with reference to FIG. 5, a description will be givenregarding the relationship between a timing at which pulsed light isemitted by the light source 107, timings at which Raman scattered lightdue to detection-target gas molecules reaches the image sensor 106, andtimings at which light is received in the floating diffusion layers ofthe image sensor 106.

FIG. 5 is a timing chart depicting a light-emission pulse, reflectedlight from each distance range portion of the target 101, controlsignals (referred to as “signal accumulation pulses”) instructing signalcharge to be accumulated in each of the floating diffusion layers (FD1to FD4) 204 to 207, and a control signal (referred to as a “draindischarge pulse”) instructing signal charge to be discharged to thedrain 202.

In the present embodiment, the measurement distance range is taken as 9m (18 m back and forth) at maximum, and four distance ranges are set ateach 2 m. The speed of light is 300,000 km per second, and therefore thetime required for light to travel back and forth over a maximum distanceof 9 m is 60 ns. If the light-emission pulse width is set as 5 ns, thetime required for one cycle is 65 ns, which is the time taken for alight component of the rear end of a light-emission pulse reflected froma location at a distance of 9 m to reach the imaging surface of theimage sensor 106. In the present embodiment, in order to exclude theeffect of Raman scattered light from afar due to the maximum distancerange, a margin period is set as 35 ns with some allowance being taken,and the total time for one cycle is set as 100 ns. Consequently, thefrequency of a light-emission pulse repeating cycle is the inversethereof of 10 MHz.

The delay time from ultraviolet excitation light being irradiated ontogas molecules to Raman scattered light being generated is generally ofthe order of picoseconds (ps), and is an order that can be more or lessignored with the measurement distance ranges of the present embodiment.Thus, a description of this delay time has been omitted; however, thetiming at which light is received by the image sensor 106 may be setwith consideration being given to the time required for Raman scatteringif necessary.

The light-emission wavelength, the light-emission pulse timing, and adrive timing for the image sensor 106 are appropriately set to optimalvalues in accordance with the distance range, the target gas, and thesensitivity of a light-receiving element. The numerical values used inthe present embodiment are exemplary and do not restrict the scope ofthe patent claims in any way.

Ultraviolet excitation light, which is 10-MHz, 5-ns width pulsed light,is irradiated onto a gas such as carbon monoxide CO, hydrogen sulfideH₂S, or ammonia NH₃ in a target space. As a result, Raman scatteredlight of a wavelength corresponding to the gas molecules is generated.The wavelengths of the Raman scattered light for excitation light of 355nm and 266 nm used in the present embodiment are as given in Table.

The light source 107 emits light-emission pulses in accordance withcontrol signals that are input from the control circuit 108. In theexample of FIG. 5, light is emitted at a timing of 0 ns, and the lightis extinguished at a timing of 5 ns. At such time, in order to excludesignal charge caused by unnecessary Raman scattering generated at aclose distance of less than 1 m outside of the measurement target range,the drain discharge pulse is set to ON. During this time, unnecessarysignal charge generated by the photodiode 203 is discharged from thedrain 202.

Raman scattered light that includes a plurality of wavelength componentsreaches the image sensor 106 being delayed according to distance asdepicted in FIG. 5. This Raman scattered light is formed into amultiplex image that is encoded and diffracted in the verticaldirection, and is converted into signal charge by the photodiode 203.

In principle, components due to unnecessary Raman scattered light from aclose distance of less than 1 m and components due to Raman scatteredlight of a distance of 1 to 3 m include crosstalk proportionate to atime corresponding to a light-emission pulse width of 5 ns. Therefore,in the present embodiment, the reception of light starts from a timingat which the crosstalk assumes a median value. Specifically, thereception of light starts from a timing of 9.17 ns obtained by adding2.5 ns, which is half of the light-emission pulse width, from a timingof 6.67 ns at which components of reflected light (1 to 3 m) start toreach the imaging surface of the image sensor 106.

The control circuit 108 sets the signal accumulation pulse for the firstfloating diffusion layer (FD1) 204 to ON at the same time as setting thedrain discharge pulse to OFF at the timing of 9.17 ns depicted in FIG.5. The signal accumulation pulse for the first floating diffusion layer204 is then set to OFF at the timing of 22.50 ns, which is when thecomponents of reflected light (1 to 3 m) attenuate and result in a 50%amount of light. Signal charge is thereby transferred and accumulated inthe first floating diffusion layer 204. Similarly, the control circuit108 sequentially sets signal accumulation pulses for the second tofourth floating diffusion layers (FD2 to FD4) 205 to 207 to ON at thetimings depicted in FIG. 5. Time-resolved signal charge that includescrosstalk proportionate to the light-emission pulse width of 5 ns isthereby sequentially transferred and accumulated in the second to fourthfloating diffusion layers 205 to 207. The control circuit 108 sets thedrain discharge pulse to ON at the timing of 62.5 ns at which the signalaccumulation pulse for the fourth floating diffusion layer 207 is set toOFF. Unnecessary signal charge due to Raman scattering generated fromafar outside of the measurement target range is thereby discharged fromthe drain 202.

By repeating the above series of operations several ten thousand timesto several hundred million times as required at the frequency of 10 MHz,signal charge of one frame of the image sensor 106 is accumulated. Thenumber of times repeated is adjusted according to the light emissionintensity of the light source 107 and the sensitivity of the imagesensor 106. The Raman scattered light is weak, and therefore, byrepeatedly performing this high-speed imaging synchronized with thelaser excitation light a considerable number of times, it is possible tocompensate for a lack of sensitivity.

In the present embodiment, the number of time resolution according tothe plurality of floating diffusion layers is four; however, it shouldbe noted that the number of time resolutions may be designed as a numberother than four in accordance with the purpose.

Next, a signal read operation subsequent to signal accumulation in theimage sensor 106 will be described with reference to FIG. 6.

FIG. 6 is a drawing depicting an example of the overall configuration ofthe image sensor 106. A region surrounded by a two-dot chain line borderin FIG. 6 corresponds to one pixel 201. The pixel 201 includes fourfloating diffusion layers 204 to 207. Signals accumulated in the fourfloating diffusion layers 204 to 207 are treated as if they were signalsof four pixels of a general CMOS image sensor, and are output from theimage sensor 106.

Each pixel 201 has four signal detection circuits. Each signal detectioncircuit includes a source follower transistor (amplification transistor)309, an FD signal-reading transistor (row selection transistor) 308, anda reset transistor 310. In this example, the reset transistor 310corresponds to the drain 202 depicted in FIG. 3, and a pulse that isinput to the gate of the reset transistor 310 corresponds to theaforementioned drain discharge pulse. The transistors are field effecttransistors formed on a semiconductor substrate, for example, but arenot restricted thereto. As depicted, one of the input terminal andoutput terminal of the source follower transistor 309 (typically thesource) and one of the input terminal and output terminal of the FDsignal-reading transistor 308 (typically the drain) are connected. Thecontrol terminal (gate) of the source follower transistor 309 isconnected to the photodiode 203. Signal charge (positive holes orelectrons) generated by the photodiode 203 is accumulated in thefloating diffusion layers 204 to 207, which are charge accumulationnodes between the photodiode 203 and the source follower transistor 309.

Although not detected in FIG. 6, the four floating diffusion layers 204to 207 are connected to the photodiode 203, and a switch is providedbetween the photodiode 203 and the floating diffusion layers. Thisswitch switches the conduction state between the photodiode 203 and thefloating diffusion layers 204 to 207 in accordance with a signalaccumulation pulse from the control circuit 108. The starting andstopping of the accumulation of signals to the floating diffusion layers204 to 207 is controlled thereby.

Signal charge accumulated in the floating diffusion layers 204 to 207 bythe abovementioned repeated operation is read out due to the gate of theFD signal-reading transistor 308 being set to ON by a row selectioncircuit 302. At such time, current that flows from a source followerpower source 305 to the source follower transistor 309 and a sourcefollower load 306 is amplified in accordance with the signal potentialof the floating diffusion layers 204 to 207. An analog signal producedby this current that is read out from a vertical signal line 304 isconverted into digital signal data by an analog-digital (AD) conversioncircuit 307 that is connected to each column. This digital signal datais read out for each column by a column selection circuit 303, and isoutput from the image sensor 106. The row selection circuit 302 and thecolumn selection circuit 303, after having performed reading for onerow, perform reading for the next row, and, similarly thereafter, readout signal charge information of the floating diffusion layers for allof the rows. After all of the signal charge has been read out, thecontrol circuit 108 resets all of the floating diffusion layers bysetting the gate of the reset transistor 310 to ON. Imaging for oneframe is thereby completed. Similarly thereafter, by repeatinghigh-speed imaging for frames, imaging for a series of frames by theimage sensor 106 is concluded.

In the present embodiment, an example of a CMOS-type image sensor 106has been described; however, the image sensor may be a CCD type, asingle photon counting-type element, or an amplification-type imagesensor (EMCCD, ICCD).

Next, a description will be given regarding processing (spectralseparation processing) for separating, into each wavelength component,each of four time-resolved images output from the image sensor 106. Thisprocessing is performed by the first signal processing unit 109 in thesignal processing circuit 112 depicted in FIG. 1.

FIG. 7 is a conceptual diagram depicting an impression of this spectralseparation processing. Images of apples are used in FIG. 7 to conveythis impression, but this is merely for convenience in order to describea multiplex image. The first signal processing unit 109 separates amultiplex image that is overlapped while being shifted in the verticaldirection with respect to each wavelength, into a plurality of images ofeach wavelength in accordance with the following procedure. This imageseparation is performed for each of the four images that are based uponthe signal charge accumulated in the floating diffusion layers 204 to207. More specifically, for each of the four images, address shifts inthe vertical direction in the wavelength component images are corrected,and w number of images F1 to Fw having the same address and not havingany overlapping due to wavelength are generated. The number w of imagesgenerated here may be any number equal to or greater than 2. Forexample, a number that is equal to or greater than 4 and equal to orless than 100 may be used.

The desired data is the spectral separation images F1 to Fw, and thatdata is represented as f. The number of spectral bands (number of bands)is w, and therefore f is data for which the image data f1, f2 . . . fwof each band is integrated. If the number of pixels in the x directionof the image data to be obtained is taken as n and the number of pixelsin the y direction is taken as m, each of the image data f1, f2 . . . fwis a collection of two-dimensional data of n x m pixels. Consequently,the data f is three-dimensional of n x m x w number of elements. When aspectroscopic element P causes a spectral image to shift one pixel at atime in the y direction for each spectral band to be obtained, thenumber of elements of data g of a captured image G that is acquired isn×(m+w−1). The data g in the present embodiment can be represented byMath. 2 below.

$\begin{matrix}{g = {{Hf} = {H\begin{bmatrix}f_{1} \\f_{2} \\\vdots \\f_{w}\end{bmatrix}}}} & \left( {{Math}.\mspace{14mu} 2} \right)\end{matrix}$

Here, f1, f2 . . . fw is data having n x m elements, and therefore,strictly speaking, a right-side vector is a one-dimensional vector of nx m x w rows and one column. A vector g is represented and calculated bybeing converted into a one-dimensional vector of n×(m+w−1) rows by onecolumn. The matrix H represents a conversion in which the vector f isintensity-modulated by means of encoding, and the components f1, f2 . .. fw are shifted one pixel at a time in the y direction and added.Consequently, H is a matrix of n(m+w−1) rows and n×m×w columns.

Here, it is assumed that images of the wavelength bands are shifted onepixel at a time, and therefore the number of elements of g is taken asn×(m+w−1); however, it is not absolutely necessary to shift the imagesone pixel at a time. The number of pixels by which shifting is performedmay be two or more pixels. The number of pixels by which shifting isperformed depends on the way which spectral bands and the number ofspectral bands in a spectral separation image F to be reconstructed havebeen set. The number of elements of g changes according to the number ofpixels by which shifting is performed. Furthermore, the spectraldirection is also not restricted to the y direction, and shifting may beperformed in the x direction. To generalize, in the case where an imageis shifted ky pixels at a time in the y direction and kx pixels at atime in the x direction with ky and kx being arbitrary natural numbers,the number of elements of data g becomes {n+kx·(w−1)}×{m+ky·(w−1)}.

Then, if the vector g and the matrix H are applied, it is seeminglypossible to calculate f by solving the inverse problem of Math. 2.However, the number of elements n×m×w of the data f to be obtained isgreater than the number of elements n(m+w−1) of the acquired data g, andtherefore this problem is an ill-posed problem and cannot be solved asit is. Thus, the signal processing circuit 112 of the present embodimentuses the redundancy of the images included in the data f, and obtains asolution using the compressed sensing method. Specifically, the data fto be obtained is estimated by solving the equation of Math. 3 below.

$\begin{matrix}{f^{\prime} = {\underset{f}{argmin}\left\{ \left. ||{g - {Hf}}||{}_{l_{2}}{+ {{\tau\Phi}(f)}} \right. \right\}}} & \left( {{Math}.\mspace{14mu} 3} \right)\end{matrix}$

Here, f′ represents the data for f that is estimated. The first termwithin the brackets of the equation above represents the amount ofdeviation between an estimation result Hf and the acquired data g, whatis referred to as the residual term. The square sum serves as theresidual term here, but an absolute value, the square root of the squaresum, or the like may serve as the residual term. The second term withinthe brackets is a regularization term (or a stabilization term) that isdescribed later on. Math. 3 means that f is obtained with the sum of thefirst term and the second term being minimized. The signal processingcircuit 112 is able to converge the solution by means of a recursiveiterative operation, and calculate the final solution f′.

The first term within the brackets of Math. 3 means a calculation thatobtains the square sum of the difference between the acquired data g andHf, for which f of the estimation process is system-converted accordingto the matrix H. ϕ(f) of the second term is a constraint in theregularization for f, and is a function that reflects sparse informationof the estimated data. As an operation, there is an effect in that theestimated data is made smooth or stable. The regularization term, forexample, may be represented by a discrete cosine transform (DCT),wavelet transform, Fourier transform, total variation (TV), or the likeof f. For example, in the case where total variation is used, it ispossible to obtain stable inferred data in which the effect of noise ofthe observed data g is suppressed. The sparsity of the measurementtarget in the space of respective regularization terms is differentdepending on the texture of the measurement target. The regularizationterm with which the texture of the measurement target becomes sparser inthe space of the regularization term may be selected. Alternatively, aplurality of regularization terms may be included in the calculation. tis a weighting coefficient, and the amount of redundant data reducedincreases (the compression ratio increases) as this value increases, andthe convergence to a solution weakens as this value decreases. Theweighting coefficient is set to an appropriate value with which fconverges to an extent and over-compression does not occur.

A calculation example using the compressed sensing indicated in Math. 3has been given here, but it should be noted that a solution may be foundusing another method. For example, it is possible to use anotherstatistical method such as a maximum likelihood estimation method, aBayesian inference method, or the like. Furthermore, the number ofspectral separation images F1 to Fw is arbitrary, and the wavelengthbands may also be arbitrarily set.

Next, 3D image generation processing by the second signal processingunit 110 depicted in FIG. 1 will be described.

FIG. 8 is a drawing depicting an overview of the 3D image generationprocessing by the second signal processing unit 110. Fourtwo-dimensional (2D) images FD1 (1 to 3 m), FD2 (3 to 5 m), FD3 (5 to 7m), and FD4 (7 to 9 m) for each distance range are obtained by means ofthe aforementioned time-resolved imaging. The second signal processingunit 110 appropriately applies a spatial high-pass enhancement filter tolevel changes for same pixel addresses between images in these 2Dimages. Thus, the effect of crosstalk generated due to thelight-emission pulse time of the pulse light source 107 having a finitelength (5 ns, for example), in other words, the effect that changes inpixel values of the same address decrease, is corrected.

Next, the type of gas is identified on the basis of a plurality ofimages for each wavelength band generated by the first signal processingunit 109. For example, in the case where excitation light having twowavelengths of 355 nm and 266 nm is used to detect three types of gas(CO, NH₃, and H₂S), according to the aforementioned Table, attention maybe paid to the components of the wavelengths of 282.1 nm, 285.9 nm,289.2 nm, 384.3 nm, 391.3 nm, and 402.7 nm. In the case where the firstsignal processing unit 109 has generated a spectral separation image foreach component of these wavelengths, by detecting the wavelength atwhich the intensity peaks, it is possible to identify the type of thegas, and it is possible to detect the concentration from the intensitylevel of the light of that peak wavelength.

Since the imaging-target space in the present embodiment is alight-transmitting body, the XY coordinates (pixel addresses) of thedata of the four images of FD1 to FD4 directly represent angles withrespect to the front direction of the horizontal direction and thevertical direction from the location of the imaging device (camera).Meanwhile, FD1 to FD4 directly correspond to distance ranges from theimaging device. Therefore, by performing a simple address conversion onthe pixels of FD1 to FD4, it is possible for three-dimensional XYZcoordinates to be calculated. A 3D spatial concentration distributionimage of a desired gas is thereby obtained.

Embodiment 2

Next, embodiment 2 of the present disclosure will be described. Similarto embodiment 1, the purpose of the imaging device in the presentembodiment is to detect the type and concentration of a gas, and it ispossible to convert a spatial distribution of the concentration of a gasinto a 3D image. The difference with embodiment 1 is that, as depictedin FIG. 9, an encoding spectroscopic element 901 in which an encodingelement and a spectroscopic element are integrated is arranged directlyin front of a time-resolved image sensor 106, and the relay opticalsystem has therefore been eliminated. Hereinafter, the presentembodiment will be described with reference to FIGS. 9 and 10 focusingon the difference with embodiment 1.

The operation up to Raman scattered light being input to the excitationlight cut filter 111 is the same as in embodiment 1, and therefore adescription thereof has been omitted.

FIG. 10 is a drawing schematically depicting a configuration of theencoding spectroscopic element 901. As depicted in (a) of FIG. 10, theencoding spectroscopic element 901 has a plurality of regions arrayedtwo-dimensionally. The regions are formed of a transparent member, andhave individually set spectral transmittances. Here, “spectraltransmittance” means the wavelength distribution of light transmittance.Spectral transmittance is represented by the function T(X), with thewavelength of incident light as X. Spectral transmittance T(X) may takea value that is equal to or greater than 0 and equal to or less than 1.In (a) of FIG. 10, 48 rectangular regions arrayed in six rows by eightcolumns are exemplified; however, in actual use, many more regions thanthis may be provided. The number thereof may be of the same level as thenumber of pixels (several hundred thousand to several ten million, forexample) of the image sensor 106, for example. In a certain example, theencoding spectroscopic element 901 may be arranged immediately above animage sensor, and each region may be arranged so as to correspond to(oppose) one pixel of the image sensor.

In (b) of FIG. 10, an example of the spatial distribution of thetransmittance of light of each of a plurality of wavelengths W1, W2 . .. Wi of a detection target is depicted. In this drawing, differences inthe shade of each region (cell) represents differences in transmittance.The transmittance increases with lighter regions, and the transmittancedecreases with darker regions. As depicted in (b) of FIG. 10, thespatial distribution of light transmittance is different depending onthe wavelength band.

In (c1) and (c2) of FIG. 10, examples of the spectral transmittance intwo regions A1 and A2 in the encoding spectroscopic element 901 aredepicted. The spectral transmittance in the region A1 and the spectraltransmittance in the region A2 are different. In this way, the spectraltransmittance in the encoding spectroscopic element 901 is differentdepending on the region. However, it is not absolutely necessary for thespectral transmittances of all of the regions to be different. Thespectral transmittances of at least some (two or more) regions fromamong the plurality of regions in the encoding spectroscopic element 901may be different from each other. In a certain example, the number ofpatterns of the spectral transmittances of a plurality of regionsincluded in the encoding spectroscopic element 901 may be the same as ormore than the number i of wavelength bands included in the targetwavelength bands. Typically, the encoding spectroscopic element 901 isdesigned such that the spectral transmittance is different in half ofthe regions or more.

FIG. 11A is a drawing for describing spectral transmittancecharacteristics in a certain region of the encoding spectroscopicelement 901. The spectral transmittance in this example has a pluralityof maximum values P1 to P5 and a plurality of minimum values in relationto the wavelengths within the target wavelength bands W. In thisexample, the maximum values for spectral transmittance are present inthe wavelength bands W2, Wi-1, and the like. In this way, in the presentembodiment, the spectral transmittance of each region has a maximumvalue in a plurality of (at least two) wavelength bands within theplurality of wavelength bands W1 to Wi.

The light transmittance of each region is different depending on thewavelength, and therefore, from within incident light, the encodingspectroscopic element 901 transmits a large number of the components ofa certain wavelength band, and does not transmit the components of otherwavelength bands to the same extent. For example, transmittance isgreater than 0.5 (50%) for the light of k number (k being an integersatisfying 2≤k<i) of wavelength bands from among i number of wavelengthbands, and transmittance is less than 0.5 (50%) for the light of theremaining i−k number of wavelength bands. In the case where the incidentlight is white light that includes all of the wavelength components ofvisible light equally, the encoding spectroscopic element 901 modulates,for each region, the incident light to light having a plurality ofdiscrete intensity peaks in relation to wavelength, and superimposes andoutputs this multi-wavelength light.

FIG. 11B, as an example, is a drawing depicting results of averaging thespectral transmittance depicted in FIG. 11A in each wavelength band W1,W2 . . . Wi. The averaged transmittance is obtained by the spectraltransmittance T(X) being integrated for each wavelength band and dividedby the width (bandwidth) of that wavelength band. In the presentspecification, a transmittance value obtained by averaging for eachwavelength band in this way is referred to as the transmittance in thewavelength band in question. In this example, the transmittanceincreases in a prominent manner in the three wavelength bands having themaximum values P1, P3, and P5. In particular, the transmittance exceeds0.8 (80%) in the two wavelength bands having the maximum values P3 andP5.

The wavelength-direction resolution of the spectral transmittance ofeach region may be set to the order of the width (bandwidth) of adesired wavelength band. In other words, from among a wavelength rangethat includes one maximum value (peak) in a spectral transmittancecurve, the width of a range that takes a value that is equal to orgreater than the average value for the minimum value closest to saidmaximum value and said maximum value may be set to the order of thewidth (bandwidth) of a desired wavelength band. In such a case, if thespectral transmittance is resolved to frequency components using aFourier transform or the like, the values of the frequency componentscorresponding to that wavelength band increase in a relative manner.

The encoding spectroscopic element 901, typically, as depicted in (a) ofFIG. 10, is divided into a plurality of regions (cells) that arepartitioned in a grid pattern. These cells have mutually differentspectral transmittance characteristics. The wavelength distribution andspatial distribution of the light transmittance of each region in theencoding spectroscopic element 901 may be the aforementioned randomdistribution or quasi-random distribution, for example.

In the case where the encoding spectroscopic element 901 is arrangednear to or immediately above the image sensor 106, the gaps (cellpitches) between the plurality of regions in the encoding spectroscopicelement 901 may substantially match the pixel pitches of the imagesensor 106. If so, the resolution of an optical image that is emittedfrom the encoding spectroscopic element 901 and encoded substantiallymatches the pixel resolution. By making light that has passed throughthe cells be incident on only one corresponding pixel, calculations thatare described later can be facilitated. In the case where the encodingspectroscopic element 901 is arranged away from the image sensor 106,the cell pitch may be reduced according to that distance.

In the example depicted in FIG. 10, a grayscale transmittancedistribution in which the transmittance of each region may take anyvalue equal to or greater than 0 and equal to or less than 1 is assumed.However, it is not absolutely necessary to implement a grayscaletransmittance distribution. For example, a binary scale transmittancedistribution in which the transmittance of each region may take eitherof the values of approximately 0 or approximately 1 may be adopted. In abinary scale transmittance distribution, each region transmits themajority of the light of at least two wavelength bands from among aplurality of wavelength bands included in the target wavelength bands,and does not transmit (blocks) the majority of the light of theremaining wavelength bands. Here, the “majority” refers to approximately80% or more.

Some cells (half, for example) from among all of the cells may bereplaced with transparent regions. Such transparent regions transmitlight of all of the wavelength bands W1 to Wi included in the targetwavelength bands, at a high transmittance of the same level (0.8 ormore, for example). In such a configuration, a plurality of thetransparent regions may be arranged in a checkered form, for example. Inother words, regions having different light transmittance due towavelength and transparent regions may be arrayed in an alternatingmanner in two array directions (the horizontal direction and thevertical direction in (a) of FIG. 10) for the plurality of regions inthe encoding spectroscopic element 901.

The encoding spectroscopic element 901 may be configured using at leastone selected from the group consisting of a multilayer film, an organicmaterial, a diffraction grating structure, and a microstructureincluding a metal. In the case where a multilayer film is used, forexample, a dielectric multilayer film or a multilayer film including ametal layer may be used. In such a case, forming is carried out suchthat at least one selected from the group consisting of the thickness,material, and layering order of the multilayer films is different foreach cell. It is thereby possible to realize spectral characteristicsthat are different depending on the cell. By using a multilayer film, itis possible to realize sharp rises and falls in spectral transmittance.A configuration in which an organic material is used may be realized bymaking the contained pigment or dye different depending on the cell, andby layering different types of materials. A configuration in which adiffraction grating structure is used may be realized by providing adiffraction structure having diffraction pitches or depths that aredifferent for each cell. In the case where a microstructure including ametal layer is used, construction may be carried out using diffractiondue to the plasmon effect.

According to the present embodiment, images having different encodinginformation for each wavelength band are formed overlapping each otheras a multiplex image on the image formation surface of the image sensor106. Different from embodiment 1, a spectroscopic element such as aprism is not used, and therefore there is no shift in the spatialdirection of the images. It is thereby possible to maintain a highspatial resolution even with a multiplex image.

The time-resolved imaging operation performed by the image sensor 106 isthe same as in embodiment 1, and therefore a description thereof hasbeen omitted. Furthermore, there is no difference with embodiment 1 alsowith respect to the signal processing apart from the acquisition of aspectral multiplex image in which there is no spatial shift. Apart theprocessing for performing vertical-direction address correction, it ispossible for a spectral separation image and a 3D image to be generatedby means of the same processing as in embodiment 1. Thus, a descriptionof the signal processing has been omitted.

In the present embodiment, it is possible to omit the relay opticalsystem by using an encoding spectroscopic element. Therefore, comparedwith the configuration of embodiment 1, it is possible to realize thesame functions with a small device.

Embodiment 3

For an imaging device of embodiment 3, a light scattering body such as aliving body is a detection target. Molecules to be detected can beidentified, and the concentration distribution thereof can bereconstructed as a 3D image. The imaging device of the presentembodiment detects the concentration distribution of oxidized hemoglobinand deoxidized hemoglobin of blood within the brain, and time changesthereof. Near-infrared light of 700 to 900 nm, known as an opticaltissue window in which absorption is relatively difficult with respectto both water and hemoglobin, may be used for a living body. Therefore,near-infrared light of this wavelength band is mainly used in thepresent embodiment.

FIG. 12 is drawing depicting a schematic configuration of the imagingdevice of the present embodiment. The imaging device is provided with anear-infrared laser pulse light source 1104, an encoding spectroscopicelement 1107, an image sensor 1106, a control circuit 1108, and a signalprocessing circuit 1112.

The near-infrared laser pulse light source 1104 irradiates a nearinfrared ray having broad spectral characteristics of 700 to 900 nmtoward brain tissue 1103, which is the detection target. Thisirradiation is repeatedly performed in short pulses in accordance withcontrol signals from the control circuit 1108.

In the present embodiment, loss due to reflection by the head surface1101 is reduced, and near-infrared light is efficiently guided into thebrain, and therefore the light source 1104 may be used in close contactwith the head surface 1101. Furthermore, reflection may be reduced usinga gel sheet, a cream, or the like. In the example depicted in FIG. 12,light-blocking plates 1105 are used in order to cut leakage light fromskin.

A portion of the irradiated short-pulse infrared light rays passesthrough the cranium 1102, is repeatedly absorbed and elasticallyscattered within the brain tissue 1103, and propagates whileattenuating. A portion thereof once again passes through the cranium1102, passes through the head surface 1101, reaches a light-receivingsurface of the image sensor 1106 via the encoding spectroscopic element1107, and photoelectric conversion is performed.

One set of the light source 1104, the encoding spectroscopic element1107, and the image sensor 1106 is drawn in FIG. 12 for simplicity;however, it should be noted that, in practice, a system may beimplemented including a plurality of sets of the light source 1104, theencoding spectroscopic element 1107, and the image sensor 1106. Aplurality of sets of the light source 1104, the encoding spectroscopicelement 1107, and the image sensor 1106 may, for example, be arrangedtwo-dimensionally at equal intervals (3 cm, for example). In the casewhere a plurality of sets of the light source 1104 and the image sensor1106 are used, a measure such as the high-speed time-divided irradiationof a laser may be implemented in order to suppress crosstalk among lightsources.

Hereinafter, an example in which one set of the light source 1104 andthe image sensor 1106 is used will be described in detail.

The same encoding spectroscopic element as in embodiment 2 can be usedfor the encoding spectroscopic element 1107. Therefore, a detaileddescription of the encoding spectroscopic element 1107 has been omitted.

In the present embodiment, different from embodiment 1 and embodiment 2,the wavelength distribution of the light emitted from the light source1104 is broad. In addition, since there is no image-forming opticalsystem, the encoding spectroscopic element 1107 is positionedimmediately in front of the image sensor 1106.

Images having different encoding information for each wavelength bandare formed overlapping each other as a multiplex image on thelight-receiving surface of the image sensor 1106. In the presentembodiment, a spectroscopic element such as a prism is not used, andtherefore there is no shift in the spatial direction of the images. Itis thereby possible to maintain a high spatial resolution even with amultiplex image.

FIG. 13 is a drawing depicting an overview of the pixel configuration ofthe image sensor 1106 in the present embodiment. The image sensor 1106in the present embodiment has a large number of floating diffusionlayers, and operates at a higher speed compared with the image sensor inembodiment 1. Hereinafter, a configuration and operation of the imagesensor 1106 will be described.

A photoelectric conversion unit (photodiode) 1203 is arranged within apixel 1201. The photodiode 1203 converts incident photons into signalelectrons. The converted signal electrons are discharged to a signalcharge discharge unit (drain) 1202, or are divided, at high speed, intoeight floating diffusion layers (FD1 to FD8) 1204 to 1211 in accordancewith control signals that are input from the control circuit 1108.

In the present embodiment, since the target is a light scattering body,the dividing is performed according to arrival time by means of thetime-resolved function of the image sensor 1106, and light is therebydivided for each optical path length of the light that is scatteredwithin the head.

FIG. 14 is a drawing depicting timings of a light-emission pulse,diffused light of each distance range, signal accumulation pulses to theeight floating diffusion layers, and a discharge pulse to the drain 1202in the present embodiment. In the present embodiment, as an example, themaximum optical path length is set as 99 cm (converted value for avacuum). The speed of light is 300,000 km per second, and therefore thetime required for light to propagate the distance of 99 cm is 3300picoseconds (ps). If the light-emission pulse width is set as 200 ps,the time required for one cycle is 3500 ps as depicted in FIG. 14, whichis the timing at which a light component of the rear end of alight-emission pulse reaches the light-receiving surface of the imagesensor 1106. In the present embodiment, in order to exclude the effectof light of a longer optical path length than the maximum optical pathlength of 99 cm, a margin period is set to 6500 ps, and the total timefor one cycle is set as 10 ns. Consequently, the frequency of alight-emission pulse repeating cycle is the inverse thereof of 100 MHz.

In the case where a plurality of light sources are used, each lightsource may be controlled so as to emit light in this margin period. Inthe case where a large number of light sources are used, the marginperiod or the light emission frequency per one light source may beadjusted as required.

The abovementioned light-emission pulse timing and drive timing for theimage sensor are examples and are not restricted to these examples.These numerical values may be appropriately set to optimal values inaccordance with the optical path length within the living body, thetarget molecules, and the sensitivity of a light-receiving element.

Hereinafter, details of the operation depicted in FIG. 14 will bedescribed. The light source 1104 emits light at a timing of 0 psdepicted in FIG. 14, and extinguishes at a timing of 200 ps inaccordance with a control signal that is input from the control circuit1108. At such time, the drain discharge pulse is set to ON in order toeliminate signal charge corresponding to leakage light that has enteredfrom outside of the head and light having an optical path length of 3 cmor less that is outside of the measurement target range. During thistime, unnecessary signal charge generated by the photodiode 1203 isdischarged from the drain 1202.

Irradiated near-infrared light, as depicted in FIG. 14, is delayedaccording to the optical path lengths and is formed into a multipleximage that is encoded and diffracted by the image sensor 1106. Thismultiplex image is converted into signal charge by the photodiode 1203.

In principle, components of light having an optical path length of lessthan 3 cm and components of light having an optical path length of 3 cmto 15 cm are included in crosstalk proportionate to a time correspondingto the light-emission pulse width of 200 ps. Therefore, in the presentembodiment, the reception of light starts from a timing at which thecrosstalk assumes a median value. Specifically, the reception of lightstarts from a timing of 200 ps obtained by adding 100 ps, which is halfof the light-emission pulse width of 200 ps, from a timing of 100 ps atwhich components of diffused light (optical path length of 3 to 15 cm)start to reach the image formation surface of the image sensor 1106.

The control circuit 1108 sets the signal accumulation pulse for thefirst floating diffusion layer (FD1) 1204 to ON at the same time assetting the drain discharge pulse to OFF at the timing of 200 psdepicted in FIG. 14. The signal accumulation pulse for the firstfloating diffusion layer 1204 is then set to OFF at the timing of 600ps, which is when the components of diffused light (optical path lengthof 3 to 15 cm) attenuate and result in a 50% amount of light. Signalcharge is thereby transferred and accumulated in the first floatingdiffusion layer 1204. Similarly, the control circuit 1108 sequentiallysets signal accumulation pulses for the second to eighth floatingdiffusion layers (FD2 to FD8) 1205 to 1211 to ON at the timings depictedin FIG. 14. Time-resolved signal charge that includes crosstalkproportionate to the light-emission pulse width of 200 ps is therebysequentially transferred and accumulated in the first to eighth floatingdiffusion layers 1204 to 1211. The control circuit 1108 sets the draindischarge pulse to ON at the timing of 3400 ps at which the signalaccumulation pulse for the eighth floating diffusion layer 1211 is setto OFF. Unnecessary signal charge due to light having an optical pathlength of 99 cm or more that is longer than the measurement optical pathlength is thereby discharged from the drain 1202.

By repeating the above series of operations several hundred thousandtimes to several trillion times as required in the frequency of 100 MHz,signal charge of one frame of the image sensor 1106 is accumulated. Thenumber of times repeated is adjusted according to the light emissionintensity of the light source 1104 and the sensitivity of the imagesensor 1106. The near-infrared diffused light that returns from withinthe brain is extremely weak light, and therefore, by repeatedlyperforming this high-speed imaging synchronized with the laser lightemission a considerable number of times, it is possible to compensatefor a lack of sensitivity.

In the present embodiment, the number of time resolutions according tothe plurality of floating diffusion layers is eight; however, it shouldbe noted that the number of time resolutions may be designed as a numberother than eight in accordance with the purpose.

FIG. 15 is a drawing depicting an example of an overall configuration ofthe image sensor 1106 in the present embodiment. A region surrounded bya two-dot chain line border in FIG. 15 corresponds to one pixel 1201.The pixel 1201 includes the eight floating diffusion layers 1204 to1211. Signals accumulated in the eight floating diffusion layers 1204 to1211 are treated as if they were signals of eight pixels of a generalCMOS image sensor, and are output from the image sensor 1106.

The operation regarding the reading of a signal after the time-resolvedimaging performed by this image sensor 1106 is the same as in embodiment1 except for there being a large number of floating diffusion layers,and therefore a description thereof has been omitted.

Next, the operation performed by the signal processing circuit 1112 willbe described. First, the wavelength band used in the present embodimentwill be described.

FIG. 16A is a graph depicting the wavelength dependency of absorptioncoefficients for oxidized hemoglobin, deoxidized hemoglobin, and water.FIG. 16B is a graph depicting the wavelength dependency of thescattering coefficient of light within biological tissue. The wavelengthband (700 to 900 nm) between the two dashed lines in the drawings isknown as an optical tissue window, and has characteristics in that theabsorption rate by the molecules within a living body is low.Near-infrared light having broad spectral characteristics of 700 to 900nm irradiated at 100-MHz, 200-ps width pulses scatters and diffusesinside the head. At such time, light is absorbed in each wavelength inaccordance with the absorption coefficients of oxidized hemoglobin anddeoxidized hemoglobin inside the living body depicted in FIG. 16A. Someof the light that propagates without being absorbed reaches thelight-receiving surface of the image sensor 1106 and is received. At awavelength of 700 to 800 nm, the absorption coefficient of deoxidizedhemoglobin is higher than the absorption coefficient of oxidizedhemoglobin, and this relationship inverts at 800 to 900 nm. Therefore,the concentration information for these molecules is included asspectral characteristics information in the scattered light within thebrain. In the present embodiment, the number of spectral separations of20 bands is set for each wavelength band width of 10 nm. It should benoted that the number of spectral separations is not restricted to thisexample, and may be appropriately set according to the requirements ofthe application applied.

A first signal processing unit 1109 in the signal processing circuit1112 separates, for each wavelength band, spectral multiplex images ofmultiple wavelengths in each of eight time-resolved images output fromthe image sensor 1106. This method is the same as in embodiments 1 and2, and therefore a description thereof has been omitted here.

Next, a second signal processing unit 1110 performs signal processingfor reconstructing a 3D image.

FIG. 17 is a flowchart depicting the flow of this signal processing.According to the aforementioned time-resolved imaging, eight 2D imagescorresponding to FD1 (3 to 15 cm), FD2 (15 to 27 cm), FD3 (27 to 39 cm),FD4 (39 to 51 cm), FD5 (51 to 63 cm), FD6 (63 to 75 cm), FD7 (75 to 87cm), and FD8 (87 to 99 cm) are output from the image sensor 1106 (stepS301). Due to the first signal processing unit 1109 performingprocessing for spectral separation, spectral images of 20 bands, forexample, are generated for each optical path length (step S302). Inother words, a total of 160 2D images of eight-optical path length,20-band diffraction are generated from imaging measurement results forone frame, per one image sensor 1106.

Next, the second signal processing unit 1110 appropriately applies ahigh-pass enhancement filter to level changes for same pixel addressesbetween images of the optical path lengths. Thus, the effect ofcrosstalk generated due to the light-emission pulse time of thenear-infrared laser pulse light source 1104 having a finite length (200ps, for example), in other words, the effect that changes in pixelvalues of the same pixel address decrease, is corrected. The secondsignal processing unit 1110 compares the result of this correction as animaging measurement result with a simulation result that is describedlater on.

FIG. 18 is a drawing depicting an example of changes over time in theamount of light that reaches one pixel. As depicted, light continuouslyreaches the one pixel in accordance with the optical path length fromthe light source 1104. An integrated value therefor is detected for eachoptical path length (FD1 to FD8). Therefore, in principle, crosstalkcomponents are included in signals for each pixel. In step S303,processing for reducing the effect of this crosstalk is performed.

Next, the flow of the simulation depicted in FIG. 17 will be described.

Within biological tissue, near-infrared light scatters considerably, andabsorption is relatively weak. Therefore, it is known that opticalphenomena that occur within biological tissue can be approximated aslight diffusion phenomena. The scattering pattern also becomes isotropicwhen the thickness of the biological tissue exceeds several mm, and theoptical energy propagates in all directions in a diffusive manner. Theintensity of scattering is represented by a scattering coefficient, andthe intensity of absorption is represented by an absorption coefficient.The inverse of these coefficients represent distance. Light propagationwithin a living body can be expressed by an optical diffusion equation.

In the 3D image reconstruction algorithm indicated in FIG. 17, first,the molecular density distribution of oxidized hemoglobin, deoxidizedhemoglobin, and the like within a living body is assumed (step S401).Next, a distribution of optical characteristic values (absorptioncoefficient and scattering coefficient) is calculated (step S402). Then,a forward problem analysis is performed, in which a model for lightpropagation within the living body is used to numerically solve thepropagation of light (step S403). In addition, components of imagesignals of each optical path length and each wavelength are calculatedunder the same conditions as the imaging conditions (step S404). Theresult thereof is compared with a result obtained by imaging measurement(step S405). If everything matches, it is determined that the assumedmolecular density distribution is correct, and that a 3D image has beenreconstructed. If the results do not match, it is determined that theassumed molecular density is incorrect, the molecular densitydistribution is re-assumed, and the processing of steps S401 to S405 isexecuted once again. The second signal processing unit 1110 repeats thisoperation until the results match in step S405 (inverse problem).

The above processing is the same as the processing performed in generaloptical diffusion tomography.

The second signal processing unit 1110 generates and outputs an imagedepicting a blood state, on the basis of obtained results andinformation on the absorption spectra of oxidized hemoglobin anddeoxidized hemoglobin depicted in FIG. 16A.

Furthermore, it is possible to simulate, in detail, the differences inthe scattering coefficient for each wavelength depicted in FIG. 16B, andby using the result thereof, the precision with which the inverseproblem is solved is improved. In the present embodiment, it is possibleto obtain, for example, a 20-band spectral image by means of spectralcompressed sensing for which a broad-band light source is used. Inaddition, by means of the time-resolved image sensor 1106, it ispossible to obtain an image (frame) having sufficient spatial samplingpoints (number of pixels). As a result, it becomes possible tosimultaneously satisfy an increase in the resolution of a desired 3Dreconstruction image and a reduction in the measurement time.

Embodiment 4

Similar to embodiment 3, an imaging device of embodiment 4 is able toreconstruct the identification of molecules to be observed and theconcentration distribution thereof as a 3D image, with a lightscattering body such as a living body serving as a target.

FIG. 19 is drawing depicting a schematic configuration of the imagingdevice of the present embodiment. The imaging device is provided with: afirst near-infrared laser pulse light source 1904, a secondnear-infrared laser pulse light source 1905, a third near-infrared laserpulse light source 1906, and a fourth near-infrared laser pulse lightsource 1907 that emit pulsed light including mutually differentwavelengths; an encoding spectroscopic element 1910; an image sensor1909; a control circuit 1911 that includes a delay adjustment circuit1915; and a signal processing circuit 1914 that includes a first signalprocessing unit 1912 and a second signal processing unit 1913. Theimaging device of embodiment 4 is different from the imaging device ofembodiment 3 in being provided with a plurality of light sources andincluding a control circuit and a delay adjustment circuit. An exampleis given in the present embodiment in which there are four lightsources, but the number of light sources is not restricted to four.

Hereinafter, differences with the operation of the imaging device ofembodiment 3 will be mainly described. The first near-infrared laserpulse light source 1904, the second near-infrared laser pulse lightsource 1905, the third near-infrared laser pulse light source 1906, andthe fourth near-infrared laser pulse light source 1907 respectivelyirradiate pulsed light of a 750-nm wavelength, pulsed light of an 800-nmwavelength, pulsed light of an 850-nm wavelength, and pulsed light of a900-nm wavelength toward brain tissue 1903, which is the detectiontarget. This irradiation is repeatedly performed in short pulses inaccordance with a control signal from the control circuit 1911.

The diffusion coefficient and scattering coefficient within the braintissue 1903 for each pulsed light are different from each other due tothe wavelengths being different. In order to eliminate this difference,the delay adjustment circuit 1915 within the control circuit 1911performs minute adjustments on the light-emission timings of the firstnear-infrared laser pulse light source 1904, the second near-infraredlaser pulse light source 1905, the third near-infrared laser pulse lightsource 1906, and the fourth near-infrared laser pulse light source 1907.

In the present embodiment, loss due to reflection by a head surface 1901is reduced, and near-infrared light is efficiently guided into thebrain, and therefore the first near-infrared laser pulse light source1904, the second near-infrared laser pulse light source 1905, the thirdnear-infrared laser pulse light source 1906, and the fourthnear-infrared laser pulse light source 1907 may be used in close contactwith the head surface 1901. Furthermore, reflection may be reduced usinga gel sheet, a cream, or the like. In the example depicted in FIG. 19,light-blocking plates 1908 are used in order to cut leakage light fromskin.

A portion of the irradiated short-pulse infrared light rays passesthrough the cranium 1902, is repeatedly absorbed and elasticallyscattered within the brain tissue 1903, and propagates whileattenuating. A portion thereof once again passes through the cranium1902, passes through the head surface 1901, reaches a light-receivingsurface of the image sensor 1909 via the encoding spectroscopic element1910, and photoelectric conversion is performed.

The layer configuration and time-resolved function of the image sensor1909 are the same as those of the image sensor 1106 of embodiment 3, andtherefore a description thereof has been omitted.

Next, the operation performed by the signal processing circuit 1914 willbe described. First, the wavelength band used in the present embodimentwill be described.

FIG. 16A is a graph depicting the wavelength dependency of absorptioncoefficients for oxidized hemoglobin, deoxidized hemoglobin, and water.FIG. 16B is a graph depicting the wavelength dependency of thescattering coefficient of light within biological tissue. Near-infraredlight having wavelengths of 750 nm, 800 nm, 850 nm, and 900 nmirradiated at 100-MHz, 200-ps width pulses scatters and diffuses insidethe head. At such time, light is absorbed in each wavelength inaccordance with the absorption coefficients of oxidized hemoglobin anddeoxidized hemoglobin inside the living body depicted in FIG. 16A. Someof the light that propagates without being absorbed reaches thelight-receiving surface of the image sensor 1909 and is received. At awavelength of 700 to 800 nm, the absorption coefficient of deoxidizedhemoglobin is higher than the absorption coefficient of oxidizedhemoglobin, and this relationship inverts at 800 to 900 nm. Therefore,the concentration information for these molecules is included asspectral characteristics information in the scattered light within thebrain. In the present embodiment, the number of spectral separations offour bands of 750 nm, 800 nm, 850 nm, and 900 nm is set. It should benoted that the number of spectral separations is not restricted to thisexample, and may be appropriately set according to the requirements ofthe application applied.

The second signal processing unit 1913 performs signal processing forreconstructing a 3D image. FIG. 17 is a flowchart depicting the flow ofthis signal processing. According to the time-resolved imaging, eight 2Dimages corresponding to FD1 (3 to 15 cm), FD2 (15 to 27 cm), FD3 (27 to39 cm), FD4 (39 to 51 cm), FD5 (51 to 63 cm), FD6 (63 to 75 cm), FD7 (75to 87 cm), and FD8 (87 to 99 cm) are output from the image sensor 1909(step S301). Due to the first signal processing unit 1912 performingprocessing for spectral separation, spectral images of four bands aregenerated for each optical path length (step S302). In other words, atotal of 32 2D images of eight-optical path length, four-banddiffraction are generated from imaging measurement results for oneframe, per one image sensor 1909.

Next, the second signal processing unit 1913 appropriately applies ahigh-pass enhancement filter to level changes for same pixel addressesbetween images of the optical path lengths. Thus, the effect ofcrosstalk generated due to the light-emission pulse time of eachnear-infrared laser pulse light source having a finite length iscorrected. The second signal processing unit 1913 compares the result ofthis correction as an imaging measurement result with a simulationresult that is described later on.

Next, the flow of the simulation depicted in FIG. 17 will be described.

In the 3D image reconstruction algorithm indicated in FIG. 17, first,the molecular density distribution of oxidized hemoglobin, deoxidizedhemoglobin, and the like within a living body is assumed (step S401).Next, a distribution of the absorption coefficient and scatteringcoefficient, which are optical characteristic values, is calculated(step S402). Then, a forward problem analysis is performed, in which amodel for light propagation within the living body is used tonumerically solve the propagation of light (step S403). In addition,components of image signals of each optical path length and eachwavelength are calculated under the same conditions as the imagingconditions (step S404). The result thereof is compared with a resultobtained by imaging measurement (step S405). If everything matches, itis determined that the assumed molecular density distribution iscorrect, and that a 3D image has been reconstructed. If the results donot match, it is determined that the assumed molecular density isincorrect, the molecular density distribution is re-assumed, and theprocessing of steps S401 to S405 is executed once again. The secondsignal processing unit 1913 repeats this operation until the resultsmatch in step S405.

The above processing is the same as the processing performed in generaloptical diffusion tomography.

The second signal processing unit 1913 generates and outputs an imagedepicting a blood state, on the basis of obtained results andinformation on the absorption spectra of oxidized hemoglobin anddeoxidized hemoglobin depicted in FIG. 16A.

In the present embodiment, it is possible to obtain a four-band spectralimage by means of spectral compressed sensing. In addition, by means ofthe image sensor 1909, it is possible to obtain an image havingsufficient spatial sampling points. As a result, it becomes possible tosimultaneously satisfy an increase in the resolution of a desired 3Dreconstruction image and a reduction in the measurement time.

As described hereinabove, according to the embodiments of the presentdisclosure, it is possible to realize an imaging device that satisfiesthe three requirements of high time resolving, high resolution, andmultiple wavelengths. In the case where a photographing target is alight-transmitting space, it is possible to perform range-gated imagingin which imaging is performed with targets being separated according todistance, and to obtain a 2D image for each wavelength corresponding tothe range of the distance from the imaging device. Furthermore, it ispossible to detect the type and concentration of gas molecules fromwavelength shifts in light due to distinctive Raman scattering orwavelength shifts in light due to fluorescence in the gas molecules. Bysimultaneously performing range-gated imaging for each of a plurality ofdistance ranges, it is possible to realize the specifying of the type ofa leaked gas and the non-contact detection of the three-dimensionaldistribution of the concentration of the gas, with a single imagingdevice.

Furthermore, in the case where the target is a light scattering body, byalso performing highly time-resolved imaging of the order ofpicoseconds, 2D images separated for each optical path length andwavelength band within a scattering body are obtained. It is possible todetect the type and concentration of molecules within the scatteringbody on the basis of these 2D images and characteristics such as thedistinctive absorption spectrum, scattering spectrum, or fluorescencespectrum of the molecules within the scattering body. In addition, it ispossible to perform reconstruction by calculating a 3D distributionimage of inside a detection-target scattering body from a plurality of2D images of each optical path length. By using image information of aplurality of wavelength bands, there is an advantage in that it ispossible to expect further improvement in the resolution of a generated3D image, on the basis of differences in the scattering coefficientwithin the scattering body for each wavelength band.

What is claimed is:
 1. A method comprising: capturing an image using animaging apparatus, the imaging apparatus including, an optical filter;and an image sensor including a first pixel and a second pixel, whereinthe optical filter includes filter regions arrayed two-dimensionally,the filter regions include a first region and a second region, awavelength distribution of an optical transmittance of the first regionhas a first local maximum in a first wavelength band and a second localmaximum in a second wavelength band that differs from the firstwavelength band, a wavelength distribution of an optical transmittanceof the second region has a third local maximum in a third wavelengthband and a fourth local maximum in a fourth wavelength band that differsfrom the third wavelength band, the third wavelength band differs fromboth the first wavelength band and the second wavelength band, the firstpixel receives light of the first wavelength band which has passedthrough the first region and light of the second wavelength band whichhas passed through the first region, and the second pixel receives lightof the third wavelength band which has passed through the second regionand light of the fourth wavelength band which has passed through thesecond region; and generating spectral separation images from the imagecaptured by the image sensor using a compressed sensing algorithm. 2.The method according to claim 1, wherein each of the first pixel and thesecond pixel of the image sensor receives light in each of wavelengthbands of the spectral separation images.
 3. The method according toclaim 1, wherein each of the first region and the second region of theoptical filter transmits light in each of wavelength bands of thespectral separation images.
 4. The method according to claim 1, whereinthe wavelength distribution of the optical transmittance of the firstregion has at least four local maxima in target wavelength bands, the atleast four local maxima including the first local maximum and the secondlocal maximum, and the third wavelength band differs from each of thewavelength bands of the at least four local maxima in the first region.5. The method according to claim 1, further comprising: irradiating atarget with pulsed light from at least one light source, the pulsedlight including components of a plurality of wavelengths.
 6. A systemcomprising: an imaging apparatus that includes, an optical filter; andan image sensor including a first pixel and a second pixel and capturingan image, wherein the optical filter includes filter regions arrayedtwo-dimensionally, the filter regions include a first region and asecond region, a wavelength distribution of an optical transmittance ofthe first region has a first local maximum in a first wavelength bandand a second local maximum in a second wavelength band that differs fromthe first wavelength band, a wavelength distribution of an opticaltransmittance of the second region has a third local maximum in a thirdwavelength band and a fourth local maximum in a fourth wavelength bandthat differs from the third wavelength band, the third wavelength banddiffers from both the first wavelength band and the second wavelengthband, the first pixel receives light of the first wavelength band whichhas passed through the first region and light of the second wavelengthband which has passed through the first region, and the second pixelreceives light of the third wavelength band which has passed through thesecond region and light of the fourth wavelength band which has passedthrough the second region; and a signal processing device that isconfigured to perform generating spectral separation images from theimage captured by the image sensor using a compressed sensing algorithm.7. The system according to claim 6, wherein each of the first pixel andthe second pixel of the image sensor receives light in each ofwavelength bands of the spectral separation images.
 8. The systemaccording to claim 6, wherein each of the first region and the secondregion of the optical filter transmits light in each of wavelength bandsof the spectral separation images.
 9. The system according to claim 6,wherein the wavelength distribution of the optical transmittance of thefirst region has at least four local maxima in target wavelength bands,the at least four local maxima including the first local maximum and thesecond local maximum, and the third wavelength band differs from each ofthe wavelength bands of the at least four local maxima in the firstregion.
 10. The system according to claim 6, further comprising: atleast one light source that irradiates pulsed light including componentsof a plurality of wavelengths to a target.
 11. A method comprising:capturing an image using an imaging apparatus, the imaging apparatusincluding an optical filter; and an image sensor including a first pixeland a second pixel and capturing an image, wherein the optical filterincludes filter regions arrayed two-dimensionally, the filter regionsinclude a first region and a second region, a wavelength distribution ofan optical transmittance of the first region has a first local maximumin a first wavelength band and a second local maximum in a secondwavelength band that differs from the first wavelength band, awavelength distribution of an optical transmittance of the second regionhas a third local maximum in a third wavelength band and a fourth localmaximum in a fourth wavelength band that differs from the thirdwavelength band, the third wavelength band differs from both the firstwavelength band and the second wavelength band, the first pixel receiveslight of the first wavelength band which has passed through the firstregion and light of the second wavelength band which has passed throughthe first region, and the second pixel receives light of the thirdwavelength band which has passed through the second region and light ofthe fourth wavelength band which has passed through the second region;and transferring the image to a signal processing device that isconfigured to perform generating spectral separation images from theimage using a compressed sensing algorithm.
 12. The method according toclaim 11, wherein each of the first pixel and the second pixel of theimage sensor receives light in each of wavelength bands of the spectralseparation images.
 13. The method according to claim 11, wherein each ofthe first region and the second region of the optical filter transmitslight in each of wavelength bands of the spectral separation images. 14.The method according to claim 11, wherein the wavelength distribution ofthe optical transmittance of the first region has at least four localmaxima in target wavelength bands, the at least four local maximaincluding the first local maximum and the second local maximum, and thethird wavelength band differs from each of the wavelength bands of theat least four local maxima in the first region.
 15. The method accordingto claim 11, further comprising: irradiating a target with pulsed lightfrom at least one light source, the pulsed light including components ofa plurality of wavelengths.
 16. The method according to claim 11,further comprising: providing, to the signal processing device, a datathat is related to a wavelength distribution of an optical transmittanceof the optical filter and used for acquiring a matrix used in thecompressed sensing algorithm.
 17. A method comprising: acquiring animage that is captured using an imaging apparatus; and generatingspectral separation images using a compressed sensing algorithm, thecompressed sensing algorithm being performed using a data of the imageand a matrix acquired based on a wavelength distribution of an opticaltransmittance of an optical filter included in the imaging apparatus,wherein the optical filter includes filter regions arrayedtwo-dimensionally, the filter regions include a first region and asecond region, the wavelength distribution of the optical transmittanceof the first region has a first local maximum in a first wavelength bandand a second local maximum in a second wavelength band that differs fromthe first wavelength band, the wavelength distribution of the opticaltransmittance of the second region has a third local maximum in a thirdwavelength band and a fourth local maximum in a fourth wavelength bandthat differs from the third wavelength band, and the third wavelengthband differs from both the first wavelength band and the secondwavelength band.
 18. The method according to claim 17, wherein each of afirst pixel and a second pixel of the image includes a component in eachof wavelength bands of the spectral separation images.
 19. The methodaccording to claim 17, wherein each of the first region and the secondregion of the optical filter transmits light in each of wavelength bandsof the spectral separation images.
 20. The method according to claim 17,wherein the wavelength distribution of the optical transmittance of thefirst region has at least four local maxima in target wavelength bands,the at least four local maxima including the first local maximum and thesecond local maximum, and the third wavelength band differs from each ofthe wavelength bands of the at least four local maxima in the firstregion.
 21. The method according to claim 17, further comprising:irradiating a target with pulsed light from at least one light source,the pulsed light including components of a plurality of wavelengths. 22.The method according to claim 17, further comprising: acquiring, inadvance to perform the compressed sensing algorithm, the matrix based ona data related to the wavelength distribution of the opticaltransmittance of the optical filter.